A video stream is a data stream, which represents a succession of video frames. A video frame is a still image, for example, a digital photograph produced by a digital camera. A video stream may be fed to a video player to display the video frames successively on a screen where it may be viewed by a viewer.
In some applications, a video stream is generated on the basis of several, that is two or more, input video streams. The input video streams may originate from separate video sources. In one example, a set of two or more video cameras are arranged to operate in parallel, each video camera generating a video stream. The video cameras may be oriented into different viewing directions so as to capture different views. A graphics processing unit may be arranged to merge the videos streams frame by frame to generate an output video stream. Each frame of the output video stream, that is, each output frame, may thus comprise video content from more than one camera.
FIG. 1 schematically shows an example of a video processing device. The video processing device 10 may for example comprise a set of video sources 12, a memory unit 14, and a processing unit 16. The processing unit 16 may also be referred to as a graphics processing unit (GPU) or a video processing unit. The set of video sources 12 may comprise two or more video sources. In the example, the set of video sources 12 comprises six video sources 12A to 12F. Each video source may for example be a video camera. The video sources 12 may be connected to the processing unit 16 via the memory unit 14. The memory unit 14 may thus act as a buffer for buffering the video streams from the video sources 12. The memory unit 14 may further be arranged to provide additional data to the processing unit 16. Such additional data may, for example, comprise geometry data. For example, each of the input video streams may have tile definitions associated with it which are the same for each image of the respective input video stream and which define the shape and position of each tile of the respective image. Furthermore, the memory unit 14 may be arranged to provide a Z buffer and a frame buffer. The memory unit 14 may be a double data rate (DDR) memory unit. The term DDR relates to transactions (e.g., read, write, or clear operations), which may be performed both on rising edges and on falling edges of a clock signal. In the example, the video sources 12 and the processing unit 16 are connected to the memory 14 via a data bus 18 (BUS). In another example (not shown), there may be a more direct connection between these units.
In operation, each of the video sources 12A to 12F may generate a respective video stream. The video streams may be fed to the processing unit 16 via the memory unit 14. The processing unit 16 may generate an output video stream on the basis of the input video streams, that is on the basis of the video streams from the video sources 12. For example, each of the video sources 12 may provide a real-time image of a certain field of view. These views may be partly overlapping and the processing unit 16 may stitch them together to form a combined real-time image, e.g., a panoramic view, a surround view, or a bird's eye view of a scene covered by the video sources 12. The video sources 12 may for example be cameras installed on a vehicle for providing a driver of the vehicle with video on a screen showing the vehicle and its current surroundings.
In one example, the memory unit 14 is arranged to store a complete frame for each of the video sources 12A to 12F, thereby enabling the processing unit 16 to generate an output frame on the basis of these input frames. At the same time, in addition to providing the set of complete frames (one complete frame per video source), the memory unit 14 may collect video data from the video sources 12 for the next frame. Furthermore, the memory unit 14 may provide memory space for the output images from the processing unit 16. The processing unit 16 may thus write the output images to the memory unit 14 where they may be accessed by another unit (not shown), e.g., by a video display device. When there are N video sources (e.g., N=6), the memory unit 14 may thus need to provide memory space for a total of P times N images from the video sources 12, wherein P is at least two.